Interservice/Industry  Training,  Simulation,  and  Education  Conference  (I/ITSEC)  2004 


Using  Computer  Games  to  Train  Information  Warfare  Teams 


Michael  J.  Cramer,  Dr.  Sowmya  Ramachandran 
Stottler  Henke  Associates,  Inc. 

San  Mateo,  CA 

cramer@stottlerhenke.com ,  sowmya@stottlerhenke.com 


Janelle  K.  Viera 
AFRL/HEAS 
Mesa,  AZ 

j  anelle.viera@mesa.afmc.af.mil 


ABSTRACT 

Information  warfare  and  security  are  crucial  to  maintaining  homeland  security.  An  important  mission  of  the 
information  warfare  force  is  to  ensure  that  secure  information  and  facilities  are  well  protected.  One  way  to  ensure 
this  is  to  try  to  gain  access  to  this  information  as  outsiders  and  see  how  well  the  practices  and  policies  designed  to 
protect  data  are  being  enforced.  Teams  of  Information  Warfare  personnel  (a.k.a.  the  Red  Teams)  are  dedicated  to 
the  mission  of  testing  the  security  of  information  and  assets  crucial  to  American  interests.  Most  such  missions 
necessitate  deception  in  order  to  test  the  extent  to  which  data  is  protected  from  strangers  and  parties  who  are  not 
trusted.  High-levels  of  stress  are  inevitable,  and  the  Red  Teams  need  to  be  highly  skilled  in  thinking  creatively 
under  such  stress.  Given  the  criticality  and  the  degree  of  danger  of  these  missions,  they  have  to  be  carefully  trained. 
For  computer-based  approaches,  providing  realistic  simulations  is  essential  for  successful  training.  Engaging  the 
trainee  emotionally  to  elicit  the  types  of  stress  responses  they  will  experience  on  real  missions  is  crucial.  3D 
computer  games  have  proved  themselves  to  be  highly  effective  in  engaging  players  motivationally  and  emotionally. 
This  effort,  therefore,  uses  gaming  technology  to  provide  realistic  simulations.  These  games  are  augmented  with 
Artificial  Intelligence  techniques  for  enabling  trainees  to  interact  with  the  simulation  using  natural  language, 
intelligent  evaluation  of  the  student’s  performance,  and  automated  after-action  review  that  allows  the  trainees  to 
assess  their  own  performance  and  provide  justifications  for  their  actions.  This  paper  describes  the  details  of  this 
approach,  providing  examples  of  the  simulations  and  after-action  reviews,  and  discusses  its  benefits  and  limitations. 
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INTRODUCTION 

A  critical  aspect  of  national  security  known  as 
Information  Warfare  (IW)  involves  actions  taken  to 
gain  and  exploit  information  about  hostile  threats 
while  also  protecting  sensitive  information  from 
getting  into  the  hands  of  the  enemy.  The  advent  of 
the  Internet  has  brought  an  explosion  in  information 
traffic.  Sharp  skills  are  required  to  be  able  to  sift 
through  volumes  of  information  efficiently,  yet 
remain  vigilant  for  crucial  data  or  evidence.  With 
warfare  shifting  from  traditional  battles  to  terrorist 
tactics,  and  the  enemy  changing  from  states  to 
amorphously  widespread,  yet  very  potent  terrorist 
organizations,  information  is  becoming  more  of  a 
crucial  asset.  Information  gathering  and  analysis  is 
critical;  and  at  least  of  equal  importance  is  the  task  of 
defending  the  security  of  one’s  own  sensitive 
information.  Creating  and  maintaining  a  highly 
trained  information  warfare  force  can  mean  the 
difference  between  victory  and  defeat. 

An  important  mission  of  the  information  warfare 
force  is  to  ensure  that  secure  information  and 
facilities  are  well  protected.  One  way  to  ensure  this  is 
to  try  to  gain  access  to  this  information  as  outsiders 
and  see  how  well  the  practices  and  policies  designed 
to  protect  information  are  being  enforced.  Teams  of 
Information  Warfare  operators  (henceforth  referred 
to  as  the  Red  Team)  are  dedicated  to  the  mission  of 
testing  the  security  of  information  and  assets  crucial 
to  American  interests.  Most  of  these  missions 
necessitate  deception  in  order  to  test  the  extent  to 
which  information  is  protected  from  strangers  and 
parties  who  are  not  trusted;  high-levels  of  stress  are 
inevitable  in  such  scenarios,  and  the  operators  need 
to  be  highly  skilled  in  thinking  creatively  in  the  face 
of  such  stress.  Given  the  criticality  and  the  degree  of 
danger  of  these  missions,  the  teams  have  to  be 
carefully  selected  and  trained.  It  is  important  to 
ensure  that  new  operators  have  the  skills  and 
personality  necessary  for  these  missions. 


THE  TRAINING  PROBLEM 

The  Red  Team  currently  undergoes  an  intense 
schoolhouse  training  program  designed  to  instruct 
new  recruits.  This  immersive  course  then  teams  the 
trainees  with  experts  and  places  them  in  real-life 
missions  to  test  their  performance  under  these 
conditions.  However,  relying  on  real  missions  for 
practice  opportunities  presents  limited  chances  for 
new  recruits  to  practice  and  test  their  skills.  Also, 
different  students  have  different  training 
requirements.  Depending  on  background,  years  of 
experience,  and  learning  speed,  some  trainees  may 
need  less  hours  of  training  than  others  to  reach  the 
same  level  of  proficiency  of  a  given  skill.  Moreover, 
different  students  need  training  in  different  areas, 
even  within  the  same  tasks.  Ideally,  the  instructor 
would  provide  a  tailored  curriculum  specific  to  a 
particular  trainee’s  needs.  However,  this  kind  of 
tailored  individualization  of  instruction  is  infeasible 
due  to  high  student  to  instructor  ratio.  Traditional 
instructor-led  classroom  training  is  inefficient 
because  it  cannot  accommodate  these  differences. 
Further,  instructors  cannot  always  monitor  an 
individual  student’s  progress  or  lack  thereof  In  a 
class  of  tens  or  hundreds  of  students,  it  is  impossible 
to  present  a  course  that  is  customized  to  everyone’s 
individual  differences.  As  a  consequence,  some 
trainees  have  to  stay  the  required  number  of  hours 
even  when  they  have  mastered  the  skills,  while  others 
don't  acquire  enough  proficiency  in  the  given  amount 
of  time. 

The  course  instructors  would  benefit  immensely  from 
a  method  or  procedure  that  could  help  them  assess 
the  competency  of  individuals  attending  the  courses  - 
prior  to  the  actual  training  -  so  that  they  can  be 
appropriately  tuned.  The  students  would  also  stand  to 
gain  much  more  from  the  coursework  if  they  are  able 
to  obtain  reasonable  expectations  of  the  skills  that 
will  be  required  of  them.  Furthermore,  there  is  an 
ongoing  need  to  provide  additional  practice 
opportunities  beyond  training,  but  live  exercises  are 
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expensive  to  eonduet  and  have  eostly  time  and  travel 
demands. 

Simulation-based  training  ean  provide  a  eost- 
effeetive  alternative  to  instruetor-led  training  where 
trainees  ean  praetiee  their  skills  under  various 
eonditions  in  realistie  but  simulated  seenarios. 
Simulations  have  been  shown  to  be  very  effeetive  for 
promoting  strategie  thinking  and  proeedural  skills 
(Sehank,  1995).  They  alleviate  the  need  for  eostly 
on-site  support  and  the  demand  for  live  training 
opportunities.  Trainees  are  free  to  partieipate  as  their 
sehedule  permits  and  need  not  all  be  present  at  one 
loeation.  Instruetors  need  not  be  present  at  all.  While 
they  are  not  fully  intended  to  replaee  live  training,  a 
simulation-based  environment  ean  be  an  effeetive 
augmentation  for  information  warfare  training  and 
also  provide  the  eapability  for  distributed  simulation. 

The  military  is  inereasingly  moving  towards  distanee 
learning  initiatives,  where  trainees  ean  partieipate 
from  remote  loeations  yet  still  work  together  as  a 
team,  thus  redueing  travel  time  and  money.  The 
Internet  provides  an  opportunity  to  reaeh  more 
personnel  in  more  geographieally  diverse  loeations 
by  providing  long-distanee  learning  and  delivery  of 
multimedia.  Unfortunately,  eurrent  Internet  training 
systems  have  been  developed  in  a  variety  of  styles 
with  little  referenee  to  sound  pedagogieal  prineiples. 
They  often  have  an  impressive  array  of  graphies, 
video,  and  animation,  but  funetionally  are  little  more 
than  simple  page-turners.  The  only  eustomization  to 
individual  students’  needs  is  aeeomplished  by  having 
the  student  self-paee  the  presentation  of  the 
information,  usually  via  navigation  through  a  hyper- 
linked,  multimedia  textbook.  When  students  require 
assistanee,  they  have  to  eommunieate  with  an  expert 
via  telephone  or  other  media.  This  ereates  a 
bottleneek  and  eauses  delays  as  students  vie  for  the 
instruetor's  time  and  await  response. 

The  solution  to  this  training  problem  is  a  simulation- 
based  distributed  training  environment  that  is  highly 
responsive  to  a  trainee’s  learning  needs.  It  should 
enable  assessment  of  a  student’s  strengths  and 
weaknesses  and  provide  rieh  and  extensive  praetiee 
opportunities  beyond  the  eourse  itself,  both  as 
prerequisite  training  and  post-deployment  refresher 
training.  An  advaneed  artifieially  intelligent  training 
system  ean  address  these  needs,  assess  student 
knowledge  in  real  time,  diagnose  defieieneies,  and 
provide  a  tailored  remedial  eourse  of  instruetion  on 
the  fly.  Sueh  intelligenee  is  erueial  to  distanee- 
leaming  programs  beeause  it  would  off-load  many  of 
the  tasks  traditionally  performed  by  instruetors. 


THE  BENEFITS  OF  USING  GAME 
TECHNOLOGY 

The  most  important  faetors  in  this  domain  are  the 
riehness  and  realism  of  the  human  interaetion,  and 
the  ability  to  think  ereatively  under  duress.  The 
simulation  absolutely  must  give  its  partieipants  both  a 
sense  of  eonfrol  and  eonsequenee,  while  suspending 
reality  to  the  largest  extent  possible.  Modem  3D 
games  -  sueh  as  those  eommonly  known  as  “first- 
person  shooters”  -  have  been  long  pushing  the 
boundaries  of  realism  by  eombining  eutting-edge 
graphies  and  real-world  physieal  modeling  with 
eomplex  eharaeter  behaviors  and  intriguing  story 
lines  eomplete  with  eonfliet  and  resolution.  Together, 
this  results  in  a  powerfully  immersive  environment 
that  engages  and  eaptivates  the  attention  of  the  player 
and  elieits  realistie  emotional  responses  from  them, 
while  at  the  same  time  stimulating  a  player’s  problem 
solving  abilities. 

Games  seem  to  motivate  players  intrinsieally  by 
stimulating  euriosity  (Thomas  and  Maeredia,  1994). 
A  sense  of  aeeomplishment  is  gained  by  a  thorough 
eomprehension  and  mastery  of  the  eauses  and  effeets 
in  the  gaming  environment;  henee  there  is  a  strong 
desire  to  understand  it.  The  theory  of  eonsttuetivism 
states  that,  by  evaluating  and  refleeting  upon  our  own 
experienees  in  that  environment,  we  eonstmet  an 
understanding  of  the  world  based  on  the  mles  and 
mental  models  we  already  have  in  plaee.  If  a  game 
ean  serve  as  an  adequate  simulation  of  the  real  world 
and  the  aetions  of  the  eonstituents  within  it  model 
their  real  world  eounterparts,  then  the  game  ean  serve 
as  a  very  effeetive  learning  tool.  This  ‘Team  by 
doing”  approaeh  is  known  to  be  an  extremely 
effeetive  edueational  tool  (Sehank,  1995). 

A  great  example  of  this  is  the  flight  simulator 
(Prensky,  2002).  Originally  eoneeived  as  an 
entertainment  deviee  for  fairs,  the  flight  simulator  is 
one  of  the  single  greatest  advaneements  in  training 
teehnology.  Pilots  ean  spend  hours  honing  their 
flying  skills  under  a  wide  variety  of  seenarios, 
varying  weather,  loeation,  flying  eonditions, 
visibility  and  even  dynamie  events  sueh  as 
meehanieal  failures.  There  is  no  danger  to  equipment 
or  lives,  and  trainees  are  free  to  try  out  any  number 
of  possible  eourses  of  aetion. 

Modem  gaming  teehnology  eontains  extensive 
support  for  both  single  and  multiplayer 
environments.  A  single  player  environment  eonsists 
of  a  single  human  player  interaeting  within  the 
simulation  via  a  set  of  eonfrols  ineluding  -  but  not 
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limited  to  -  a  computer  keyboard  and  mouse.  The 
simulation  consists  of  a  pre-constructed  world  that 
behaves  according  to  a  model  based  on  real  rules  of 
physics  and  contains  one  or  more  artificially 
intelligent  non-player  characters  (henceforth  known 
as  NPC’s)  acting  within  that  world.  Each  character 
(both  players  and  NPC’s)  have  a  physical 
embodiment  in  the  game  world  (such  as  a  pawn  or 
avatar)  and  can  only  move  and  interact  in  accordance 
with  the  physical  rules  of  that  embodiment. 
Multiplayer  environments  have  more  than  one  human 
player  and  any  number  of  NPC’s  (or  none  at  all) 
participating  over  some  networking  medium.  These 
environments  also  support  any  number  of  omniscient 
observers  having  no  physical  effect  upon  the  world 
but  able  to  observe  the  various  internal  and  external 
states  of  objects  within  that  world. 

Single  player  environments  can  be  used  to  provide 
initial  training  at  an  individual  level.  The  NPC’s  in 
these  scenarios  play  the  role  of  tester  by  assuming  the 
roles  of  the  “opposing”  characters  in  the  scenarios  - 
roles  that  would  normally  subsume  the  time  of  the 
instructors.  This  is  ideal  for  self-paced  training.  The 
trainee  is  then  free  to  attempt  any  number  of  scenario 
permutations  as  his  or  her  time  will  allow.  Moreover, 
intelligent  tutors  partake  in  the  scenario,  taking  note 
of  the  tasks  that  the  student  performs  poorly  as  well 
as  documenting  the  student’s  strengths.  They  report 
this  information  to  both  the  student  and  the 
instructors  in  the  form  of  after  action  review,  giving 
the  instructors  advance  knowledge  of  the  student  and 
giving  the  student  an  idea  of  what  the  instructor  will 
expect. 

Multiplayer  environments  can  support  more 
advanced  training  where  teams  of  trainees,  experts, 
and  instructors  can  participate  in  a  simulation 
simultaneously  from  remote  locations.  The  tutor  will 
take  a  back  seat  in  this  approach,  making  only 
observations,  while  coaching  and  after-action  review 
is  performed  ad  hoc  by  the  instructors.  This  saves  in 
travel  time  and  expense  as  well  as  provides  a 
convenient  medium  for  continued  training  well  after 
the  coursework  is  complete. 

DEVELOPING  A  GAME-LIKE  SIMULATION 

The  game  is  the  cornerstone  of  computer-based 
training;  it  is  important  to  assure  that  it  will  be 
immersive,  realistic,  and  educational.  Therefore  a 
compelling  prototype  was  the  primary  thrust  of  our 
initial  effort.  Phase  I  of  USAF  SBIR  F33615-03-M- 
6349.  It  was  designed  to  assess  the  feasibility  of 
developing  a  system  that  satisfies  the  needs  thus  far 


discussed.  We  implemented  this  prototype  taking  into 
account  the  necessary  design  considerations  of  a  fully 
functional  system  -  a  simulation-based  intelligent 
tutoring  system  (ITS)  to  complement  the  existing 
course  -  targeted  for  completion  at  the  end  of  Phase 
II  of  this  effort. 

For  this  prototype,  we  focused  on  the  single  player 
aspect  of  the  game,  where  trainees  interact  with 
simulated  characters.  We  drafted  a  training  scenario 
and  storyline  using  actual  case  studies  and  guidelines 
gathered  from  our  subject  matter  experts,  the 
graduates  and  instructors  of  this  in-house  course. 
Then  we  devised  game  parameters  and  rules  to 
establish  motivation  and  direction  for  the  game  play. 
We  used  a  commercial  game  engine.  Unreal 
Tournament  2003^'^,  and  its  associated  construction 
tools  to  provide  the  physical  modeling  and  graphical 
rendering  of  our  simulated  environment.  Finally,  we 
played  the  game  through  all  the  encounters  to  test  the 
validity  and  authenticity  of  each  encounter. 

The  Scenario 

The  scenario  begins  outside  a  military  facility  where 
the  trainee  and  his  assistant  (an  NPC)  will  attempt  to 
infiltrate  a  high  security  network  using  a  wide  variety 
of  covert  strategies  and  persuasion  techniques.  The 
two  are  faced  with  a  series  of  seven  encounters, 
where  they  will  be  confronted  by  hostile  characters 
(NPCs)  that  will  likely  be  suspicious  of  their 
activities. 

Figure  1  shows  a  screen  shot  from  the  beginning  of 
the  game,  just  as  the  player  has  entered  the  facility.  If 
the  trainee  successfully  persuades  the  character  into 
performing  a  desired  action  (such  as  letting  them 
pass  through  or  escorting  them  to  the  servers)  they 
move  on  to  the  next  encounter.  Once  the  trainee 
successfully  proceeds  through  all  encounters  in  the 
scenario,  then  he  has  succeeded  in  the  lesson  plan.  If 
a  trainee  is  denied  access,  or  arouses  a  high  level  of 
suspicion,  he  is  deemed  to  have  failed  in  the  lesson. 

In  each  encounter,  there  are  opportunities  for  the 
trainee  to  increase  or  decrease  suspicion  and  anxiety 
levels.  The  suspicion  level  refers  to  the  overall  sense 
of  mistrust  that  the  facility  has  for  the  player  and  is 
influenced  by  the  interaction  of  the  player  with  each 
of  the  characters  throughout  the  scenario.  It  is 
important  to  maintain  minimal  suspicion  levels  as  it 
will  determine  the  overall  success  or  failure  of  the 
mission.  The  anxiety  level  refers  to  the  level  of 
tension  and  worry  that  the  trainee’s  assistant  is 
experiencing.  The  player  needs  to  maintain  minimal 
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anxiety  in  his  assistant,  as  it  will  affect  her  ability  to 
function  smoothly  and  efficiently. 


Figure  1:  View  at  start  of  game. 


Simulated  characters  form  the  primary  obstacle  to  the 
completion  of  the  scenario.  They  will  confront  the 
trainee  about  the  details  of  his  presence  in  a  given 
area,  or  about  the  actions  in  which  he  is  attempting  to 
engage.  The  trainee  must  successfully  persuade  them 
that  his  presence  and  activities  are  normal  and 
legitimate.  Failure  to  do  so  can  cause  immediate 
mission  failure  or  make  subsequent  encounters  much 
more  difficult  and  stressful.  Given  the  limited  scope 
of  the  prototype,  the  interactions  between  the  trainer 
and  the  other  characters  are  scripted  with  branching 
to  enable  multiple  paths  through  the  script  depending 
on  player  responses.  At  each  decision  point,  the 
character  with  whom  the  player  is  interacting  will  ask 
a  question  or  perform  an  action  requiring  a  response. 
The  player  is  presented  with  a  numbered  set  of 
response  choices  any  of  which  he  can  choose,  or  he 
can  opt  not  to  select  anything.  If  a  player  does  not 
select  an  option  within  a  preconfigured  amount  of 
time,  a  “no  response”  is  assumed  and  is  considered  a 
perfectly  valid  decision  (though  not  necessarily  a 
good  or  bad  one).  The  player’s  choice  at  each  point 
determines  the  subsequent  behavior  of  the  NPC  and 
can  affect  the  suspicion  and  anxiety  level  of  the 
game.  Figure  2  shows  a  screen  shot  of  one  of  these 
decision  points  with  a  set  of  response  options 
displayed. 

In  addition  to  the  question  interface,  the  game  also 
features  a  heads-up  display  (HUD)  text  overlay  that 
contains  dynamically  updated  game  information: 
current  suspicion  level,  anxiety  level,  time  elapsed 
since  the  start  of  the  game,  and  any  useful  inventory 
items  collected  from  the  surroundings.  These  are 
visible  in  the  upper  left,  lower  right,  upper  right,  and 


lower  left  comers  of  Figure  2,  respectively.  This  is  a 
reference  guide  to  help  the  students  make  appropriate 
choices  and  get  immediate  feedback  regarding  the 
effects  of  his  decision.  The  display  is  updated  in  real 
time.  Whenever  a  player’s  decision  raises  or  lowers 
suspicion  (or  anxiety)  the  display  is  updated 
immediately  so  that  a  player  gets  the  appropriate 
positive  or  negative  reinforcement.  In  addition,  a 
player  can  monitor  time  as  well,  since  he  or  she  has  a 
limited  time  to  complete  the  objective. 


Figure  2:  Question  interaction  from  first 


encounter. 


A  player  is  evaluated  on  his  ability  to  keep  suspicion 
levels  to  a  minimum,  and  his  ability  to  manage  the 
anxiety  level  of  his  assistant.  When  the  suspicion 
level  reaches  a  maximum  limit,  the  game  ends  and 
the  mission  fails.  Also,  high  suspicion  levels  may 
lead  to  more  skeptical  behaviors  from  the  scenario 
characters,  making  it  even  harder  for  a  player  to 
complete  his  mission.  High  anxiety  levels  lead  to 
more  erratic  behavior  from  the  player’s  assistant, 
which  in  turn  can  induce  suspicion  if  not  handled 
appropriately.  Further,  high  anxiety  impedes  the 
assistant’s  ability  to  work  quickly,  opening  more 
opportunities  for  dubious  characters  to  interject 
further  probing  inquiries. 

The  player  controls  movement  by  keyboard  and 
mouse  (or  alternatively,  a  joystick).  The 
configuration  of  these  controls  assumes  the  standard 
Unreal  Tournament  2003^'^  default.  As  with  any 
game,  the  controls  can  be  remapped  according  to  the 
trainee’s  preferences.  Players  can  look  around  by 
moving  the  mouse  around  (commonly  called 
“mouseview”  in  gaming  terminology)  and  move 
backwards,  forwards  and  sideways  (strafing)  by 
pressing  a  key  or  mouse  button,  depending  on  how 
that  trainee  has  his  or  her  keys  mapped.  Pawns  do  not 
have  much  physical  capabilities  in  the  prototype 
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(such  as  jumping,  crouching,  and  running)  due  to  the 
limited  scope  of  the  project  and  the  fact  that  the 
scenario  does  not  call  for  it.  Table  1  provides  a  brief 
description  of  the  encounters. 


Table  1:  Encounter  descriptions. 


Description 

Possible  Effects 

1 

The  trainee  is  confronted 
by  the  guard  at  the  front 
desk  who  questions  the 
vaiidity  of  the  trainee’s  iD. 

Heightened  suspicion, 
increased  anxiety  if  the 
trainee  cannot 
downplay  the  obvious 
poor  quality  of  the  ID. 

2 

The  trainee  is  questioned 
casuaiiy  by  an  affable 
escort  who  is  lamenting  the 
current  state  of  security. 

Heightened  suspicion 
if  the  trainee  appears 
too  secretive  and 
standoffish  about  their 
work. 

3 

The  trainee  and  assistant 
are  questioned  by  a 
technical  person  who 
happens  upon  them  in  a 
corridor. 

Increased  suspicion 
and  anxiety  if  the 
trainee  does  not  allow 
the  assistant  to  handle 
the  more  technical 
questions  to  which  she 
is  better  suited  to 

answer. 

4 

The  trainee  must  allay  his 
assistant’s  anxiety  about 
having  her  bag  scanned 
through  an  X-ray  machine. 

Increased  suspicion 
and  anxiety  if  the 
trainee  does  not 
remain  calm  and 
attract  attention. 

5 

The  assistant  is  probed  by 
the  second  escort  if  much 
attention  was  attracted  in 
the  second  encounter. 

This  encounter  only 
occurs  if  the  previous 
one  resulted  poorly. 

The  trainee  must  again 
defuse  the  situation  or 
more  suspicion  will 
result. 

6 

The  trainee  must  provide 
distractions  to  the  escort 
while  his  assistant 
performs  the  security 
breach. 

Increased  suspicion, 
failure  of  mission  if 
poor  distraction 
techniques  are 
chosen. 

System  Architecture 


simplified  architecture  for  the  prototype  is  depicted 
in  Figure  3. 


Character 
,  Behaviors 

Question 

Responses 

Figure  3:  Overview  of  the  prototype 
architecture. 


As  mentioned  previously,  we  used  the  Unreal 
Tournament  2003^'^  game  engine  for  building  the 
game  environment.  It  comes  equipped  with  CAD-like 
modeling  software  which  was  used  to  build  the  world 
and  a  proprietary  scripting  language  which  we  used 
to  construct  the  character  behavior,  game  rules,  and 
evaluation.  The  game  engine  renders  all  graphics  and 
animation,  plays  sounds  and  controls  game  play.  At 
the  heart  of  the  system  is  the  interaction  manager 
which  controls  the  game  flow,  sequencing  each 
encounter  and  signaling  each  of  the  simulated 
characters  and  evaluates  the  current  state  of  the 
game.  The  interaction  manager  oversees  decisions  by 
prompting  the  player  when  possible  decision  points 
are  reached.  The  heads-up  display  provides  a  window 
into  the  game  play  by  presenting  a  first-person  view 
of  the  environment,  response  options  and  review  at 
the  end  of  the  game. 


The  bulk  of  the  work  effort  for  this  prototype  was 
dedicated  to  developing  a  realistic  game  environment 
and  engaging  storyline.  Flowever,  it  is  important 
when  developing  a  prototype  to  account  for  the 
requirements  of  the  target  system  and  the  prototype 
architecture  should  reflect  that  to  a  reasonably  large 
extent.  To  this  end,  we’ve  implemented  a  minimal 
architecture  satisfying  the  demands  of  the  scenario 
we  have  put  forth,  but  allowing  for  additional 
extensibility  and  refinement  without  impacting 
anything  in  place  thus  far.  While  the  architecture  of 
the  Phase  II  product  is  beyond  the  scope  of  this 
paper,  some  aspects  of  intelligent  tutoring  specific  to 
this  domain  are  discussed  in  the  next  section.  The 


AUGMENTING  THE  GAME  WITH 
AUTOMATED,  REFLECTIVE 
AFTER-ACTION  REVIEW 

The  training  course  augmentation  needs  to  be  a 
realistic  simulation-based  training  system  that 
provides  assessment  information  to  instructors.  Due 
to  the  high  unpredictability  and  randomness  of  this 
domain,  it  is  difficult  -  if  not  impossible  -  to  devise  a 
single  evaluator  that  instructs  the  student  in  a  manner 
that  is  in  accordance  with  all  domain  expertise.  In 
other  words,  a  lot  of  situations  present  cases  where  a 
decision  is  not  clearly  right  or  wrong.  Each  choice 
has  a  number  of  outcomes  possessing  variable 
desirability,  based  on  factors  ranging  from  the 
strengths  and  weakness  of  the  student  to 
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unpredictable  dynamic  factors  in  the  scenario.  In 
these  cases  there  is  no  absolute  right  or  wrong,  only 
varying  consequences  of  particular  actions,  and  the 
student  should  have  some  idea  of  each  and  every  one 
of  these  outcomes.  Therefore,  the  tutoring  strategy  in 
which  they  are  most  interested  is  reflective  after- 
action  review.  We  accomplish  this  by  taking  a 
multiple  agents-based  approach  to  review.  Rather 
than  conducting  evaluation  from  a  signal,  evaluator,  a 
number  of  agents  are  selected  to  conduct  the  review 
(which  will  be  explained  in  the  next  paragraph). 
After  review  is  complete,  a  model  of  the  trainee’s 
performance  is  constructed  using  the  tasks  and  skills 
that  the  trainee  performed  exceptionally  well,  as  well 
as  those  that  the  trainee  performed  poorly. 

The  Panel  of  Judges  Approach 

To  accommodate  this  wide  range  of  feedback 
permutations,  we’ve  devised  an  innovative 
evaluation  system  that  mimics  the  effect  of  multiple 
instructors.  In  addition  to  the  implementation  of  a 
large  number  of  evaluation  “behaviors”,  each  of 
these  behaviors  will  also  have  personality  attributes 
that  affect  the  behavior  accordingly.  For  example, 
while  two  evaluators  are  analyzing  a  student’s 
actions  in  one  particular  encounter,  one  evaluation 
entity  could  be  very  strict  and  demanding,  “playing  it 
by  the  book”  while  the  other  could  be  more  lenient 
and  focused  on  positive  reinforcement  and  creative 
thought.  Then,  during  the  after-action  review,  the 
student  is  presented  with  a  “panel  of  judges”  that 
each  weigh  in  on  the  student’s  performance.  These 
agents  can  behave  outside  their  character  at  random 
times  to  avoid  predictability,  if  necessary.  In 
addition  to  a  composite  score  representing  the 
student’s  overall  performance,  each  evaluator  reports 
its  own  viewpoint  on  the  student’s  decisions.  This  is 
not  restrained  to  general  dispositions,  either.  A 
particular  evaluator  may  have  a  bias  toward  a 
particular  problem-solving  or  stress  management 
approach  and  may  thus  appear  more  favorable  when 
a  student  takes  that  approach  and  more  critical  when 
a  student  does  not. 

To  enrich  the  variety  of  scenario  evaluations,  the 
tutor  will  assign  personality  types  to  the  judges 
randomly  at  the  start  of  a  scenario.  This  implies  that 
repeated  runs  of  the  same  scenario  would  be  different 
as  agent  personalities  will  be  instantiated  differently. 
The  same  can  be  said  for  the  NPC’s  working  within 
the  scenario.  For  example,  one  run  might  include  a 
gullible  escort  who  may  not  question  the  trainee 
much,  while  another  run  might  include  a  skeptical 
escort  who  will  grill  the  trainee  thoroughly.  This 


makes  it  possible  to  get  several  different  types  of 
evaluations  from  a  single  scenario  specification. 

Student  Modeling 

This  element  maintains  a  dynamic  model  of  the 
student.  This  information  is  crucial  for  determining 
the  next  instructional  step,  for  scenario  selection,  and 
character  selection.  In  the  past  we  have  used  various 
metrics  such  as  percentages  of  correct  usage  of  a 
principle  and  Bayesian  probability  inference  to 
measure  skill  mastery.  The  technique  depends  on  the 
organization  of  the  principles,  and  the  degree  of 
interdependence  between  the  principles.  While  it  is 
important  to  “score”  the  student  for  grading  and 
accounting  purposes,  for  this  domain,  it  is  more 
important  to  minimize  the  importance  on  placing  a 
score  on  the  student’s  performance,  and  place  more 
emphasis  on  providing  detailed  feedback  to  the 
instructors  so  that  they  can  make  the  appropriate 
decisions  as  to  where  the  student  needs  more  work. 

At  the  end  of  the  Phase  I  development  effort,  we 
demonstrated  a  preliminary  version  of  the  prototype 
to  representatives  from  the  user  group  and  received 
various  suggestions  which  have  been  incorporated 
into  the  Phase  II  design.  While  most  of  the 
suggestions  were  related  with  the  fidelity  of  the 
graphics,  which  can  be  addressed  quite  easily,  there 
was  one  suggestion  that  stood  out  for  its  technical 
implications.  The  users  require  that  the  interactions 
of  the  user  with  the  other  non-playing  characters  not 
be  restricted  to  multiple-choice  responses.  We  have 
folded  in  this  requirement  into  the  Phase  II  prototype 
by  proposing  to  use  Latent  Semantic  Analysis  and 
other  template -based  techniques  to  allow  trainees  to 
communicate  with  the  world  through  free-form 
inputs  in  either  natural  English  or  a  close  subset. 

ANALYSIS  OF  THE  BENEFITS  AND 
COSTS  OF  THE  APPROACH 

The  project  is  in  the  second  phase  under  an  Air  Force 
SBIR  contract  and  development  is  estimated  to  occur 
during  FY2005  and  FY2006.  The  development  team 
is  currently  projected  to  consist  of  2-4  programmers, 
2-3  artists,  a  storyboard  consultant  and  a  project 
manager.  Provisions  have  also  been  made  for  the 
development  of  a  scenario  authoring  toolkit,  whereby 
subject  matter  experts  can  tweak  scenarios  and 
behaviors  to  other  requirements.  At  the  completion 
of  this  effort,  the  system  can  be  maintained  by  one 
technical  consultant  per  distribution  site  and  a  small 
team  of  instructors  (at  any  location)  to  oversee 
training. 


2004  Paper  No  1729  Page  8  of  10 


Interservice/Industry  Training,  Simulation,  and  Education  Conference  (I/ITSEC)  2004 


One  potential  intangible  cost  is  the  lack  of  human 
involvement  as  the  ITS  assumes  a  more  prominent 
role  in  the  student  curriculum.  As  more  of  this 
instruction  becomes  automated,  the  human 
instructors  become  estranged  and  oblivious  of 
individual  students’  needs.  However,  it  is  important 
to  note  that  this  system  is  not  intended  to  be  a 
replacement  for  human  tutelage,  but  rather  an 
augmentation.  Since  a  few  instructors  are  relied 
upon  to  teach  many  students,  the  instructors  must 
utilize  the  time  that  they  do  get  to  spend  wisely,  and 
thus  address  the  students’  most  glaring  needs.  In  this 
way,  the  ITS  is  used  as  a  pre-assessment  and  post¬ 
follow-up  tool  that  can  automate  some  of  the 
instructor’s  more  mundane  and  rudimentary  tasks 
and  formulate  a  general  overview  of  the  student’s 
strengths  and  weaknesses. 

Training  via  simulated  environments  and  ITS  can  be 
repeated  ad  nauseum  or  until  the  student  has  a  good 
idea  of  the  concepts  and  principles  that  will  be 
expected  of  her.  This  way,  when  actual  one-on-one 
training  begins  with  human  instructors,  students  have 
a  better  idea  of  what  to  expect  and  can  anticipate 
being  better  prepared  for  their  encounters.  This  will 
facilitate  learning  and  make  optimal  use  of  both  the 
instructor’s  and  student’s  time  and  require  less 
training  in  the  long  run.  Furthermore,  students  and 
instructors  can  participate  at  any  location,  relieving 
them  of  costly  and  time-consuming  travel 
requirements.  Instructors  will  also  have  the 
capability  to  oversee  a  large  number  of  trainees 
simultaneously  by  playing  the  role  of  omniscient 
observer.  They  need  not  participate  in  the  scenario, 
but  can  quietly  watch  behind  the  scenes  as  players 
interact  with  one  another  and  with  simulated 
characters.  This  further  allows  them  to  build  their 
own  conceptions  of  student  strengths  and  weaknesses 
without  costly  one-on-one  personal  interaction. 
When  necessary,  they  can  intervene  in  any  scenario 
and  give  necessary  feedback  and  advice. 

We  will  conduct  studies  to  evaluate  the  quantitative 
and  qualitative  effectiveness  of  the  tutor  provisional 
on  the  availability  of  students.  The  precise  evaluation 
methodology  will  depend  on  the  number  of  trainees 
available  for  the  study,  and  the  amount  of  content 
authored  by  the  subject  matter  experts.  Quantitative 
studies  require  larger  number  of  participants  and 
long-term  use  requiring  extensive  content  to  provide 
any  conclusive  results  about  training  effectiveness.  If 
the  conditions  are  not  supportive  of  such  a  large- 
scale  study,  we  will  conduct  qualitative  studies  that 
require  fewer  resources  in  terms  of  participants  and 
content. 


CONCLUSION 

The  ITS  we  have  developed  combines  gaming 
technology  with  ITS  technology,  resulting  in  a 
training  tool  that  is  at  once  engaging,  adaptive,  and 
flexible.  Games  have  always  had  the  capacity  to 
engage  players  motivationally  and  emotionally.  They 
recently  have  begun  to  attract  the  attention  of 
developers  of  computer-based  training  programs. 
However,  games  by  themselves  have  limited 
potential  as  training  programs  unless  they  can 
provide  a  degree  of  automated  coaching,  tutoring, 
and  feedback.  Without  these  elements,  games,  like 
simulations,  will  suffer  from  the  problem  that 
trainees  will  play  games  without  necessarily  learning 
much  from  them.  Learning  requires  structure  and 
feedback  to  complement  inspiration.  ITS  technology 
can  enhance  the  training  potential  of  games  by 
providing  these  elements.  The  proposed  effort  is  a 
step  towards  marrying  the  two  technologies. 

Our  system  has  an  innovative  reflective,  after-action 
review  component  for  leading  trainees  through  an 
assessment  of  their  performance  in  a  domain  where 
there  are  no  clear  right  or  wrong  answers.  The 
objective  of  the  after  action  review  is  to  let  the 
trainee  reflect  on  various  perspectives  and  draw  their 
own  conclusions  as  well  as  provide  the  instructors  a 
complete  overview  of  student  performance  without 
generalizing  to  a  strict  set  of  principles  and 
guidelines.  We  believe  this  will  provide  a  very  cost- 
effective  solution  to  pre-training  evaluation  and 
remedial  learning. 

RELATED  WORK 

Our  prototype  is  an  innovative  combination  of 
gaming  and  intelligent  tutoring  technology.  However, 
the  idea  for  using  games  as  a  training  tool  is  not  new. 
The  U.S.  Army  has  been  using  games  as  a  means  of 
training  and  recruitment  for  some  time  now. 
America’s  Army,  Full  Spectrum  Command,  and 
AWE  are  three  examples  of  how  gaming  has  come  to 
the  forefront  of  training  methodologies. 

America’s  Army 

America’s  Army  is  an  Army-based  3D  computer 
game  designed  to  depict  the  authentic  U.S.  Army 
experience.  This  includes  the  units  and  equipment 
used  in  real  Army  maneuvers,  and  multiplayer 
missions  that  involve  direct  action,  surveillance, 
reconnaissance  and  combat  search  and  rescue. 
Players  conform  to  the  laws  of  land  warfare.  Army 
values  (honor,  duty  and  integrity)  and  realistic  rules 
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of  engagement  as  they  work  together  to  accomplish 
their  mission,  receive  evaluation  and  progress  up  the 
ranks  on  the  way  to  becoming  a  Green  Beret.  This 
game  was  conceptualized  by  the  Army  as  a  recruiting 
tool,  but  has  developed  into  a  very  popular, 
massively  multiplayer  game.  As  such,  it  has  very 
little  in  the  way  of  actual  training  and  instructor 
feedback  and  is  used  primarily  for  entertainment 
purposes. 

Full  Spectrum  Commaud 

Full  Spectrum  Command  is  a  PC-based  training  aid 
that  models  the  command  and  control  of  a  U.S.  Army 
Light  Infantry  Company  in  an  urban  environment.  As 
the  captain  of  the  company,  the  player  receives 
orders  for  a  given  scenario,  organizes  her  platoons, 
and  coordinates  the  actions  of  her  soldiers  during  the 
engagement.  The  scenarios  were  designed  to  develop 
critical  cognitive  skills  such  as  tactical  decision¬ 
making,  resource  management  and  adaptive  thinking. 
These  scenarios  are  focused  on  asymmetric  threats 
within  peacekeeping  and  peace-enforcement 
operations.  Each  scenario  was  developed  with  the  aid 
of  the  US  Army  Infantry  Center  in  Fort  Benning, 
Georgia  and  subject  matter  experts  from  the 
Singapore  Armed  Forces  to  ensure  both  military  and 
pedagogic  fidelity.  This  game  focuses  more  on 
training  than  America’s  Army  and  emphasizes 
learning  by  doing,  though  it  still  lacks  a  strong 
instructor  feedback  model  and  course  framework. 

Asymmetric  Warfare  Environment  (AWE) 

Asymmetric  Warfare  Environment  is  a  massively 
multiplayer  simulation  that  will  be  used  by  military 
personnel  to  train  troops  in  urban  situations  before 
they  are  airlifted  to  a  battle  zone.  The  goal  is  to  train 
soldiers  mentally  for  what  can  be  a  very  trying 
experience.  The  game  does  not  concern  itself  with  AI 
since  there  are  so  many  players  participating;  it  is 
designed  to  bring  players  together  from  all  over  the 
world.  Its  rich  content  is  delivered  through  a  large. 


private  broadband  network  owned  by  the  U.S.  Army. 
This  medium  allows  them  to  bring  ideas  and  share 
experiences  in  a  realistic  environment.  This  is  much 
more  closely  in  line  with  our  vision;  however,  we  see 
the  addition  of  AI  as  a  valuable  tool  for  offloading 
some  of  the  instructional  and  feedback  duties. 
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